DOCUMENT RESUME 



ED 464 087 

AUTHOR 

TITLE 



PUB DATE 
NOTE 
PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 
ABSTRACT 

factors that affect college students’ self-rating of change in problem 
solving and critical thinking abilities after 4 years of college. Based on 
the available literature, these six factors were selected for investigation: 
(1) Scholastic Assessment Test (SAT) scores; (2) a socioeconomic indicator; 

(3) sex; (4) academic self-concept; (5) academic major; and (6) activities in 
the last year of college. Data were from the 1986-1990 Cooperative 
Institutional Research Project surveys of first-time, full-time students who 
enrolled as first-year students in 1986. A sample of 1,056 students provided 
complete information on all variables, and path analysis was used to test the 
proposed model. Data were analyzed using multiple regression procedures, and 
causal effects suggested by the model were estimated with ordinary least 
squares procedures. The results support five of the hypothesized seven paths 
and identify four additional significant paths. In the estimated model, the 
activities in which the student participated in the last year of college had 
the greatest significant direct effect. Academic self-concept was found to 
have both significant indirect and direct effects. The indirect effects were 
mediated only through activities, as academic major had no influence on 
ratings of self-change. As expected, prior achievement was found to have only 
an indirect effect on ratings of self -change. (Contains 4 tables, 2 figures, 
and 20 references.) (Author/SLD) 



TM 033 767 

Appling, Susan A. 

A Model of Influences on Students’ Self-Rating of Change in 
Problem Solving and Critical Thinking Abilities after Four 
Years of College. 

2001 - 00-00 

2 Op . ; Notations on figures may not reproduce well. 

Reports - Research (143) 

MF01/PC01 Plus Postage. 

♦Change; College Entrance Examinations; *College Students; 
♦Critical Thinking; Extracurricular Activities; Higher 
Education; Individual Development; * Influences; Majors 
(Students) ; *Problem Solving; Sex Differences; Socioeconomic 
Status 

♦Academic Self Concept 



The purpose of this study was to examine the influence of 6 




Reproductions supplied by EDRS are the best that can be made 
from the original document. 



TM033767 



t 



oo 

o 

rt* 

\o 

rt* 



@ 



A Model of Influences on Students’ Self-rating of Change in 
Problem Solvins and Critical Thinkins Abilities 

W w 1 

After Four Years of College 



Susan A. Appling 
University of Memphis 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 

S • 



US DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
DUCATIONAL RESOURCES INFORMATION 
/ CENTER (ERIC) 

x<his document has been reproduced as 
received from the person or organization 
originating it. 

Minor changes have been made to 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 

1 



• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 




2 



2 



ABSTRACT 

The purpose of this study was to examine the influence of six factors that affect college 
students’ self-rating of change in problem solving and critical thinking abilities after four years 
of college. Critical thinking ability has been identified as a major outcome of higher education. 
Researchers have suggested that the college experience in general is more important in the 
development of critical thinking skills than are individual components of that experience. The 
literature is inconclusive regarding which particular aspects of the college experience contribute 
to development of these skills. Based on the available literature, the following six factors were 
selected for investigation: SAT scores, a socioeconomic indicator, sex, academic self-concept, 
academic major, and activities in the last year of college. 

Data used for this study were from the 1986 - 1990 Cooperative Institutional Research 
Project (CIRP) surveys of first-time full-time students who enrolled as freshmen in 1986. After 
selection of the specific variables for this study, a sample of 1056 students who provided 
complete information on all variables was used. Path analysis was used to test the proposed 
model. Data were analyzed using multiple regression procedures. Causal effects suggested by the 
model were estimated with ordinary least squares procedures. 

The results supported five of the hypothesized seven paths, and identified four additional 
significant paths. In the estimated model, activities in which the student participated during the 
last year of college had the greatest significant direct effect. Academic self-concept was found to 
have both significant indirect and direct effects. The indirect effects were mediated only through 
activities, as academic major had no influence on ratings of self-change. As expected, prior 
achievement was found to have only an indirect effect on ratings of self-change. 
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INTRODUCTION 

Critical thinking ability, composed of a cluster of intellectual skills, has been identified as a 
major outcome of higher education (Pascarella, 1999). Factors that influence problem solving 
and critical thinking abilities in college students include interactions with faculty and peers, 
residence, and out-of-class experiences, among others (Inman & Pascarella, 1998; Terenzini, et. 
al., 1996; Whitt, et ah, 1999). In addition, college effects research has identified three primary 
findings: college attendance positively impacts cognitive development; special instruction can 
selectively target improvement in critical thinking skills; and primary study in different content 
areas has little impact on development of these skills (Inman & Pascarella, 1998). These 
researchers suggested that the college experience in general is more important in the 
development of critical thinking skills than is individual components of that experience. They 
have stated that the literature is inconclusive regarding which particular aspects of the college 
experience contribute to cognitive growth. As such, a variety of factors were selected to 
examine their influence, both direct and indirect, on students’ self-rating of change in problem 
solving and critical thinking abilities after four years of college. 

Previous research supports the concept that individual involvement in the educational 
system will likely have a positive impact on development of cognitive skills (Astin, 1984; Inman 
& Pascarella, 1998; Pace, 1979). Astin (1984) proposed that a student’s progress toward a goal 
is a function of the time and energy spent on activities related to that goal. Therefore, it would 
be expected that participation in particular types of activities would enhance problem solving and 
critical thinking abilities. Activities such as students tutoring each other, discussing course 
content with each other, and working on class projects together are cooperative learning 
techniques that have been found to facilitate active learning (Lambiotte et al., 1987). Since active 
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learning facilitates cognitive development (Anaya, 1999), these activities were thought to be 
important in the development of problem solving and critical thinking skills. 

Biglan (1973) described a three-dimensional classification schema for academic subject 
matter areas. The three dimensions he described include “hard” versus “soft,” “pure” versus 
“applied,” and “life systems” versus “nonlife systems.” However, only the “hard” versus “soft” 
aspect was used in this study. Biglan has described “hard” majors as those in the more 
“scientific” areas such as engineering, physics, statistics, anatomy and orthopaedics. “Soft” 
majors were described as those in humanities and education, such as anthropology, social work, 
economics, history and philosophy. While Pascarella and Terenzini (1991) concluded that 
major had little impact on cognitive development, others have suggested there is a relationship 
between these variables (Feldman & Newcomb, 1969), or that a lack of influence of major may 
be the result of methodological decisions or classification systems of academic departments 
(Feldman, Smart & Ethington, 1999). Since the relationship between major area of study and 
the development of problem solving and critical thinking skills is unclear, further investigation is 
warranted. 

Academic self-concept has previously been operationally defined as a student’s self- 
rating of academic ability and intellectual self-confidence (Ethington, 1990). When estimating a 
model of student persistence, Ethington identified significant direct effects of academic self- 
concept on degree aspirations, expectations for success, desire for recognition, and political 
goals. While self-concept had no direct effect on persistence, a significant total effect was found 
with level of degree aspirations being the mediating variable. In addition, she found that 
academic self-concept was the dominant mediating variable for the indirect influence of prior 
achievement. Her results suggested that students’ expectations were influenced by their goals 
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and perceptions of their own abilities. Further investigation of the relationship between student 
perception of change in their abilities after college and academic self-concept is warranted. 

Based on the previous research cited, six factors were selected to investigate their impact 
on self-ratings of change in critical thinking and problem solving abilities. The purpose of this 
study was to estimate the influence of these factors on the self-reported change in problem 
solving and critical thinking skills after four years of college. 

Causal Model 

A path analytical model was proposed and is presented in Figure 1. The model was 
conceptualized utilizing the perspective taken by Eccles, et. al. (1983) in their development of a 
model of achievement behaviors. The present study focused only on the psychology component 
of the Eccles model. Variables were selected based on their expected relationship to the 
dependent variable, self-reported change in problem solving and critical thinking abilities. The 
arrows indicate hypothesized influences, all expected to be positive. The hypothesized effect 
was thought to be zero in those locations where no arrow is drawn. The first block of variables 
was composed of exogenous variables. This block included SAT scores, a socioeconomic (SES) 
indicator, and sex. These variables are correlated for reasons unanalyzed in this study. SAT 
score, the sum of verbal and math SAT scores, was used as a measure of prior achievement. SES 
was defined as the sum of mother’s education level, father’s education level and joint income. 

Sex was included to examine the effects of gender on academic self-concept. The exogenous 
variables were used to control for any differences that may have been present upon entering 
college and are consistent with Eccles (1983) model. No direct influence of these exogenous 
variables on self-rating of change in problem solving and critical thinking abilities was predicted. 
Any influence they might exert was thought to be indirect through intervening variables. 
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Figure 1. Proposed Model of Influences on Self-rating of Change in Problem Solving and Critical Thinking 
Ab dsties 




Endogenous variables in the model included: academic self-concept, composed of self- 
ratings of academic ability and intellectual self-confidence upon entering college; a selection of 
activities participated in during the last year of college; major after four years of college, 
classified according to Biglan (1973) as either “hard” or “soft”; and the self-rating of change in 
problem solving and critical thinking abilities after four years of college. Academic self-concept 




7 



7 



is a construct within the psychological component of the Eccles achievement model (1983). The 
variables of major and activities participated in during the last year of college were brought into 
the present model based on models used in the study of college effects on students (Pascarella & 
Terenzini, 1991). 

As illustrated in Figure 1, each exogenous variable was thought to have a direct influence 
on academic self-concept, but only indirect influences on subsequent variables. Likewise, 
academic self-concept was thought to have direct influence on both major and on participation in 
activities during the last year of college. Both of these factors were thought to have direct 
influence on students’ self-rating of change in problem solving and critical thinking abilities. 
However, since both major and activities during the last year of college are in the same block, no 
causal relationships were specified between these variables. Each subsequent variable was 
thought to be a mediating variable for preceding ones. 

METHOD 

Data 

Data used for this study were from the 1986 - 1990 Cooperative Institutional Research 
Project (CIRP) surveys of first-time full-time students who enrolled as freshmen in 1986. The 
Higher Education Research Institute at the University of California, Los Angeles conducted this 
survey, (see Astin, 1982) The data set contained responses given to a subset of the survey items 
by a sample of 2,891 students who attended a four-year institution and did not transfer between 
1986 and 1990. After selection of the specific variables for this study, a sample of 1,056 
students who provided complete information on all variables was used. A description of the 
variables used and how they were constructed is provided in Table 1. 



Table 1. Variable Definitions 



Prior Achievement 
Socioeconomic Status 



Sex 

Academic Self-Concept 



Major (Biglan Categories) 
Activities Last Year 



Self-rating of Change 



SAT verbal and math scores were summed after standardizing (alpha = 

.723). 

Mother’s and father’s education level (8 levels with 1 = “grammar school 
or less” to 8 = “graduate degree”) and combined parental income (14 levels 
with 1 = “less than $6,000” to 14 = “$150,000 or more”) were summed 
after standardizing (alpha = .725). 

1 = male; 2 - female 

Student’s self-rating (freshman year) of academic ability and self- 
confidence were summed after standardizing (alpha = .640). Each was 
coded 1 = “lowest 10%”, 2 = “below average”, 3 = “average”, 4 = “above 
average”, and 5 = ‘highest 10%” 

Major after 4 years of college, in Biglan categories. Recoded such that 1 = 
soft and 2 = hard. 

Scale computed by summing across students’ report of participation in 7 
particular activities during the last year of college. These activities 
included: worked on independent research project; took interdisciplinary 
course; discussed course content with students; worked on group projects in 
class; tutored another student; did extra (unassigned) work for a course; 
studied with other students. These 7 items were coded 1 = “not at all”, 2 = 
“occasionally”, and 3 = “frequently”. 

Students’ self-rating of change in problem solving skills and self-rating of 
critical thinking ability were summed after standardizing (alpha = .639). 

Each was coded 1 = “much weaker”, 2 = “weaker”, 3 = “no change”, 4 = 
“stronger”, and 5 = “much stronger”. 



Analysis 

Path analysis was used to test the proposed model. Data were analyzed using multiple 
regression procedures with SPSS 8.0 and causal effects suggested by the model were estimated 
with ordinary least squares procedures using GEMINI (Wolfle and Ethington, 1985). Because of 
the large sample size, the critical alpha level was set at .01. 



RESULTS 

Correlations, means, and standard deviations are reported in Table 2. The parameter 
estimates are presented in Table 3 as standardized beta regression coefficients. Direct, indirect 
and total causal effects are given in Table 4. The results of the analyses indicated that 
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Table 2. Correlations, Means and Standard Deviations (N=1056) 





1 


2 


3 


4 


5 


6 


7 


1 . Prior Achievement 


1.0000 














2. SES 


0.3000 


1.0000 












3. Sex 


-0.1420 


-0.0490 


1.0000 










4. Academic Self -Concept ■ 


0.5180 7 


0.1300 


-0.1890 


1.0000 








5. Major 


0.2780 


0.0500 


-0.2120 


0.2950 


1.0000 






6. Activities in Last Year 


0.0250 


0.0790 


0.0350 


0.1360 


0.1180 


1.0000 




^ Self-rating: oTChange 


0.0400 


0:0460 


0700210 — 


—0:1350- 


-210:0940 


0:2540— 


— LQ0.003 


^ Means 


0.0951 


0.3775 


1.4300 


0.2650 


1.3134 


14.3996 


0.0756 


Standard Deviations 


1.7456 


2.3650 


0.5000 


1.7061 


0.4641 


2.3001 


1.6959 



Table 3. Estimation of Model of Influences on Problem Solving/Critical Thinking 







DEPENDENT 


VARIABLES 






Academic 


Major 


Activities in 


Self-rating 


INDEPENDENT 

VARIABLES 


Self-Concept 


(Biglan) 


Last Year 


of Change 


Prior Achievement 


.510* 


.171* 


-.085 - 


-.042 


SES 


-.029 


-.033 


.084* 


.025 


Sex 


-.118* 


-.155* 


.061 


.023 


Academic Self-Concept 
Major 

Activities in Last Year 




.181* 


.181* - 


.112* 

.049 

.232* 



* p < .01 



Table 4. Direct, Indirect, and Total Effects on Self-rating of Change in 
Problem Solving & Critical Thinking Abilities 





Direct 


Indirect 


Total 


INDEPENDENT VARIABLES 


Effects 


Effects 


Effects 



Prior Achievement 


-.042 


.0714* 


.030 


SES 


.025 


.0132 


.038 


Sex 


.023 


-.0126 


.010 


Academic Self-Concept 


.112* 


.0506* 


.163* 


Major 


.049 




.049 


Activities in Last Year 


.232* 




.232* 



*p<.01 
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multicoUinearity was not a problem in this model. The estimated model is presented in Figure 2. 
Arrows are drawn for statistically significant paths only, with beta weights provided. 

The theoretical model hypothesized direct influences of both major and activities in the last year 
of college on self-rating of change in problem solving and critical thinking abilities. In addition, 
the other variables in the model were hypothesized to have only indirect effects. However, 
academic self-concept and participation in activities in the last year of college thought to be 
related to enhancement of problem solving and critical thinking abilities both had significant 
direct effects on self-rating of change. Activities in the last year of college included working on 
independent research, participation in interdisciplinary courses, involvement in cooperative 
learning strategies, and others. Participation in these activities in the last year of college had the 
strongest significant effect on students’ self-rating of change in problem solving and critical 
thinking abilities (Beta = .232), while students’ academic self-concept had about half of that 
magnitude of influence (Beta = . 1 12). The positive effects from these influences indicated that 
having more involvement during the last year of college and a higher academic self-concept as a 
freshman directly enhanced improvement of the cognitive abilities of problem solving and 
critical thinking. The hypothesized direct effect of major on self-rating of change in problem 
solving and critical thinking abilities was not supported. The results indicated that the variables 
in the model explain about 8% of the variance in self-rating of change in problem 
solving and critical thinking abilities. The results supported theories by Pace (1979), Astin 
(1984), and Pascarella (1985b) regarding the quality of effort and involvement of the student in 
cognitive development. 

Further examination of the proposed model and the actual findings revealed other 
differences. Other significant direct effects found that were not hypothesized include prior 
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Figure 2. Actual Model of Influences on Self-rating of Change in Problem Solving and Critical Thinking 
Abilities 




achievement and sex on major (Beta = .171 and -.155, respectively), and SES on activities in the 
last year of college (Beta = .084). These findings suggested that the higher level of prior 
achievement, the more likely a student was to select a “hard” major, as defined by Biglan (1973). 
“Hard” majors included those in the physical sciences and mathematics. In addition, males were 
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more likely to select “hard” majors than were females. The direct effect of SES on activities in 
the last year of college indicated that the higher the student’s SES level, the more they were 
likely to participate in the specific activities during their last year. This finding was surprising, 
as those activities selected were not thought to be SES dependent. While it was proposed, SES 
did not have a significant direct effect on academic self-concept. 

Academic self-concept had the same influence on both major and activities in the last 
year of college (Betas = .181). The higher the self-concept, the more likely a student was to 
select a “hard” major and the more likely a student was to participate in activities. As proposed, 
a significant direct effect of prior academic achievement on academic self-concept was found 
(Beta = .510). Those students with a high level of prior academic achievement, as measured by 
the SAT verbal and math scores, had a higher academic self-concept. A direct effect of sex on 
academic self-concept was also found (Beta = -.155), with males found to have a higher 
academic self-concept than females. 

Both prior academic achievement and academic self-concept were found to have 
significant indirect effects on self-rating of change in problem solving and critical thinking 
abilities (.0714 and .0506, respectively). In addition to indirectly influencing self-rating of 
change in problem solving and critical thinking abilities through activities in the last year of 
college, academic self-concept was found to be the predominant mediating variable for the 
indirect influence of prior academic achievement. Academic self-concept has been previously 
shown to be a dominant mediating variable for the indirect influence of prior academic 
achievement (Ethington, 1990). 

The total effects found in this study represent the combined direct and indirect effects of 
the variables in the model. Activities in the last year of college had the greatest total impact of 
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any variable on self-rating of change in problem solving and critical thinking abilities. In 
addition, academic self-concept also had significant total effects due to it’s significant direct and 
indirect effects on self-rating of change. Although a significant indirect effect of prior 
achievement was found, it was not strong enough to result in significant total effects. 

CONCLUSIONS 

The purpose of this study was to examine the influence of factors that affect college students’ 
self-rating of change in problem solving and critical thinking abilities after four years of college. 
The results supported five of the seven hypothesized paths, and identified four additional 
significant paths. Activities participated in during the last year of college and academic self- 
concept were the only variables that had significant direct effects and significant total effects on 
change in problem solving and critical thinking abilities. Prior achievement and academic self- 
concept had significant indirect effects on the dependent variable, with academic self-concept 
mediating the effect of prior achievement. The variables in this model explained only about 8% 
of the variance in self-rating of change in problem solving and critical thinking abilities after 
four years of college. Therefore, further research is indicated to reveal other factors that 
influence students’ self-rating of change in these skills after college. 

In the estimated model, activities in which the student participated during the last year of 
college had the greatest significant direct effect and total effect on student self-rating of change. 
This finding, that individual participation of the student in the educational system facilitated 
improvement in the cognitive skills of problem solving and critical thinking, is consistent with 
Pace’s theoretical conception of quality of effort (1979). The significant positive relationship of 
participation in these activities and self-reported improvement in critical thinking and problem 
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solving skills is also consistent with findings of other researchers regarding quality of student 
effort during college. (As tin, 1984; Pasearella, 1985; Inman and Pasearella, 1998). The activities 
in the last year of college selected for analysis in this study included cooperative learning 
strategies such as working on group projects, tutoring other students, and discussing course 
content with other students, as well as independent activities. The results of this study also lend 
support to the concept that use of cooperative learning strategies facilitates students’ active 
involvement in the learning process, resulting in improvement of cognitive skills, like problem 
solving and critical thinking (Anaya, 1999). 

Academic self-concept was proposed to have only indirect effects on self-ratings of 
change in problem solving and critical thinking abilities, mediated by major (Biglan, 1973) and 
participation in activities in the last year. However, academic self-concept was found to have 
both significant direct and indirect effects, as well as being the mediating variable for the effect 
of prior achievement on self-ratings of change in problem solving and critical thinking abilities 
after college. The indirect effects of academic self-concept were mediated only through activities 
in the last year of college, as major had no influence on ratings of self-change in problem solving 
and critical thinking abilities. As noted previously, there has been some debate regarding the 
nature of the relationship between major field of study and development of cognitive skills, such 
as critical thinking and problem solving (Feldman & Newcomb, 1969; Pasearella & Terenzini, 
1991). The present finding, that major had no influence on self-ratings of change in problem 
solving and critical thinking, is consistent with the notion that it is the college experience as a 
whole rather than the particular field of study that influences the development of cognitive skills 
(Inman and Pasearella, 1998; Pasearella and Terenzini, 1991). 
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While no other variables had significant effects on students’ self-rating of change in 
problem solving and critical thinking abilities after four years of college, other significant paths 
were identified in the model. With regard to major as classified by Biglan (1973), academic self- 
concept, prior achievement, and sex had significant effects on selection of a “hard” major versus 
a “soft” major. A positive relationship was found between academic self-concept and selection 
of a “hard” major, such as one in the basic sciences, statistics, or engineering. The higher the 
academic self-concept, the more likely a student was to select a “hard” major. Likewise, a 
positive relationship was found between prior achievement and selection of a “hard” major. A 
student with a higher level of prior achievement, in this case measured by the combined verbal 
and math scores of the SAT, the more likely that student would be to select a “hard” major. A 
negative relationship was found between sex and selection of a “hard” major, as males (coded 1) 
were more likely to select a “hard” major than were females (coded 2). The “hard” disciplines 
are more quantitative in nature and therefore involve more mathematics. This finding is 
consistent with similar research that suggests males are more likely to select a major in a 
quantitative field than are females (Ethington, Smart, & Pascarella, 1988; Ethington & Wolfle, 
1988). In addition, a negative relationship was also found between sex and academic self- 
concept, indicating males, on average, had a higher academic self-concept than did females. 

Two other variables had significant effects in the estimated model. Prior achievement 
had a positive relationship on academic self-concept, suggesting students with a higher level of 
prior achievement also had a higher academic self-concept. This is consistent with previous 
research findings that prior academic achievement has a positive influence on academic self- 
concept (Ethington, 1990). As noted previously, the effect of prior achievement was strong 
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enough to have a significant indirect effect on self-ratings of change in problem solving and 
critical thinking abilities, being mediated by academic self-concept, 

Finally, there was a significant effect of SES on activities participated in during the last 
year of college. The direct effect of SES on those activities indicated that the higher the 
student’s SES level, the more they were likely to participate in the specific activities during their 
last year. This finding was surprising, as those activities selected were not thought to be SES 
dependent. That is, the activities selected did not require additional financial resources. 

However, one could postulate that students of lower SES levels may have less “free” time to 
participate in such activities, as they may spend those hours working. While Astin (1993) found 
a negative relationship between full-time work and academic performance, the relationship of 
employment status while in college and change in problem solving and critical thinking s kills has 
not been fully analyzed. 

In summary, the results found in this study further support previously published results 
relating to outcomes of college and quality of student effort (Pace, 1979, Astin, 1984; Pascarella, 
1985; Inman and Pascarella, 1998). In this study, the strongest effects on self-reported change in 
problem solving and critical thinking abilities after four years of college were the result of 
participation in selected activities during the last year of college and the student’s academic self- 
concept. The variables in this study accounted for only 8% of the variance in problem solving 
and critical thinking skills. Therefore, more research is needed to further elucidate other factors 
that impact not only problem solving and critical thinking abilities, but also the student’s 
perception of change in these abilities after four years of college. 
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